Update FV3-JEDI ctest case to match MPAS-JEDI case #238

SamuelDegelia-NOAA · 2024-11-27T21:14:48Z

Description

This PR unifies the FV3-JEDI and MPAS-JEDI test cases in RDASApp. Now the FV3-JEDI test case and ctests are valid at the same time (2024-05-27, 00:00 UTC) which allows for better comparison between GSI (RRFSv1), FV3-JEDI, and MPAS-JEDI. Thanks to @chunhuazhou for providing the background files for this case!

The case data are now linked in to RDASApp/fix/expr_data/fv3_2024052700 through init.sh (similar to the MPAS-JEDI test data). The link_fv3jedi_expr.sh script is then added to create a new experiment directory during the build called RDASApp/expr/fv3_2024052700. New bumploc files and template jobs for running fv3-jedi and bump are also added for running this new case. The reference data are also updated for the new case.

Staged data and tools are also added for running GSI-based EnVar for this test case. The GSI configuration is taken from the RRFS_A retros on WCOSS2 with some minor changes including disabling dBZ state variables, disabling the hybrid (now using pure EnVar), a different mesonetuselist, and turning off ext_sonde.

Given that some developers are still using the old FV3-JEDI test for validation, the old case is still linked into RDASApp/fix/rrfs-test-data/rrfs-data_fv3jedi_2022052619 and RDASApp/bundle/rrfs-test-data/rrfs-data_fv3jedi_2022052619. As such, the old test data can still be accessed for validation experiments using RDASApp/scripts/setup_experiment.sh. But only the new test data is linked into RDASAppp/fix/expr_data and used for the ctests.

Issue(s) addressed

No issue was opened for this PR.

Dependencies (if applicable)

None.

Checklist

I have performed a self-review of my own code.
I have run rrfs tests before creating the PR (if applicable).
I have staged the new FV3-LAM test data on RDAS_DATA
I have staged the GSI test data on RDAS_DATA

guoqing-noaa · 2024-11-27T21:28:00Z

rrfs-test/scripts/setup_experiment.sh

@@ -154,6 +154,7 @@ if [[ $GSI_TEST_DATA == "YES" ]]; then
  sed -i "s#@YOUR_PATH_TO_GSI@#${YOUR_PATH_TO_GSI}#g" ./run_gsi.sh
  sed -i "s#@SLURM_ACCOUNT@#${SLURM_ACCOUNT}#g"       ./run_gsi.sh
  sed -i "s#@MACHINE_ID@#${MACHINE_ID}#g"             ./run_gsi.sh
+  sed -i "s#ANAL_TIME@#2022052619#g"                  ./run_gsi.sh


Sorry for the picky comment, Lines 154-157 may be merged into one sed command.

I don't mean we have to modify this, but if we would like to modify it, here is a reference for running multiple sed commands in one line:
https://github.com/NOAA-EMC/rrfs-workflow/blob/62b7336628738c5d9f94a87a80aa864e15e0adc5/scripts/exrrfs_fcst.sh#L69

guoqing-noaa

LGTM.

We can run CI tests when it is ready.

SamuelDegelia-NOAA · 2024-11-30T03:00:00Z

I just pushed an update to use the larger set of obs for the new ctests instead of the single ob yamls used previously. I had to remove the ATMS obs since FV3-JEDI expects fields with different names than our restart files have. I also had to recreate the domain check obs since the FV3 domain is slightly different from our MPAS domain.

This commit also switches the FV3-JEDI ctest from LETKF to GETKF (split mode) to better match our MPAS-JEDI configuration.

rrfsbot · 2024-11-30T14:17:33Z

PASSED on hera

started build_and_test on hera at UTC time: Sat Nov 30 13:19:05 UTC 2024
finished at UTC time: Sat Nov 30 14:17:32 UTC 2024

Test project /scratch1/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/238/build/rrfs-test
    Start 2: rrfs_fv3jedi_2024052700_getkf_observer
    Start 5: rrfs_mpasjedi_2024052700_getkf_observer
    Start 1: rrfs_fv3jedi_2024052700_Ens3Dvar
    Start 4: rrfs_mpasjedi_2024052700_Ens3Dvar
    Start 7: rrfs_mpasjedi_2024052700_bumploc
    Start 8: rrfs_bufr2ioda_msonet
1/8 Test #8: rrfs_bufr2ioda_msonet .....................   Passed   26.47 sec
2/8 Test #7: rrfs_mpasjedi_2024052700_bumploc ..........   Passed  116.85 sec
3/8 Test #1: rrfs_fv3jedi_2024052700_Ens3Dvar ..........   Passed  138.33 sec
4/8 Test #2: rrfs_fv3jedi_2024052700_getkf_observer ....   Passed  156.97 sec
    Start 3: rrfs_fv3jedi_2024052700_getkf_solver
5/8 Test #5: rrfs_mpasjedi_2024052700_getkf_observer ...   Passed  308.23 sec
    Start 6: rrfs_mpasjedi_2024052700_getkf_solver
6/8 Test #4: rrfs_mpasjedi_2024052700_Ens3Dvar .........   Passed  424.37 sec
7/8 Test #3: rrfs_fv3jedi_2024052700_getkf_solver ......   Passed  388.39 sec
8/8 Test #6: rrfs_mpasjedi_2024052700_getkf_solver .....   Passed  1356.44 sec

100% tests passed, 0 tests failed out of 8

Label Time Summary:
mpi            = 2916.04 sec*proc (8 tests)
rdas-bundle    = 2916.04 sec*proc (8 tests)
script         = 2916.04 sec*proc (8 tests)

Total Test time (real) = 1664.70 sec

workdir: /scratch1/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/238

rrfsbot · 2024-11-30T14:21:12Z

FAILED on hercules

started build_and_test on hercules at UTC time: Sat Nov 30 13:15:49 UTC 2024
finished at UTC time: Sat Nov 30 14:21:11 UTC 2024

Test project /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/238/build/rrfs-test
    Start 2: rrfs_fv3jedi_2024052700_getkf_observer
    Start 5: rrfs_mpasjedi_2024052700_getkf_observer
    Start 1: rrfs_fv3jedi_2024052700_Ens3Dvar
    Start 4: rrfs_mpasjedi_2024052700_Ens3Dvar
    Start 7: rrfs_mpasjedi_2024052700_bumploc
    Start 8: rrfs_bufr2ioda_msonet
1/8 Test #8: rrfs_bufr2ioda_msonet .....................   Passed   33.47 sec
2/8 Test #1: rrfs_fv3jedi_2024052700_Ens3Dvar ..........   Passed  143.65 sec
3/8 Test #7: rrfs_mpasjedi_2024052700_bumploc ..........   Passed  160.48 sec
4/8 Test #2: rrfs_fv3jedi_2024052700_getkf_observer ....   Passed  257.57 sec
    Start 3: rrfs_fv3jedi_2024052700_getkf_solver
5/8 Test #4: rrfs_mpasjedi_2024052700_Ens3Dvar .........***Failed  412.32 sec
6/8 Test #3: rrfs_fv3jedi_2024052700_getkf_solver ......   Passed  358.96 sec
7/8 Test #5: rrfs_mpasjedi_2024052700_getkf_observer ...   Passed  629.64 sec
    Start 6: rrfs_mpasjedi_2024052700_getkf_solver
8/8 Test #6: rrfs_mpasjedi_2024052700_getkf_solver .....   Passed  1406.93 sec

88% tests passed, 1 tests failed out of 8

Label Time Summary:
mpi            = 3403.01 sec*proc (8 tests)
rdas-bundle    = 3403.01 sec*proc (8 tests)
script         = 3403.01 sec*proc (8 tests)

Total Test time (real) = 2037.52 sec

The following tests FAILED:
	  4 - rrfs_mpasjedi_2024052700_Ens3Dvar (Failed)
Errors while running CTest
Output from these tests are in: /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/238/build/rrfs-test/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

workdir: /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/238

rrfsbot · 2024-11-30T14:21:45Z

PASSED on jet

started build_and_test on jet at UTC time: Sat Nov 30 13:16:25 UTC 2024
finished at UTC time: Sat Nov 30 14:21:45 UTC 2024

Test project /lfs5/BMC/wrfruc/rrfsbot/PRs_RDASApp/238/build/rrfs-test
    Start 2: rrfs_fv3jedi_2024052700_getkf_observer
    Start 5: rrfs_mpasjedi_2024052700_getkf_observer
    Start 1: rrfs_fv3jedi_2024052700_Ens3Dvar
    Start 4: rrfs_mpasjedi_2024052700_Ens3Dvar
    Start 7: rrfs_mpasjedi_2024052700_bumploc
    Start 8: rrfs_bufr2ioda_msonet
1/8 Test #8: rrfs_bufr2ioda_msonet .....................   Passed   41.65 sec
2/8 Test #7: rrfs_mpasjedi_2024052700_bumploc ..........   Passed  177.87 sec
3/8 Test #1: rrfs_fv3jedi_2024052700_Ens3Dvar ..........   Passed  181.39 sec
4/8 Test #2: rrfs_fv3jedi_2024052700_getkf_observer ....   Passed  222.84 sec
    Start 3: rrfs_fv3jedi_2024052700_getkf_solver
5/8 Test #5: rrfs_mpasjedi_2024052700_getkf_observer ...   Passed  417.53 sec
    Start 6: rrfs_mpasjedi_2024052700_getkf_solver
6/8 Test #4: rrfs_mpasjedi_2024052700_Ens3Dvar .........   Passed  608.89 sec
7/8 Test #3: rrfs_fv3jedi_2024052700_getkf_solver ......   Passed  434.95 sec
8/8 Test #6: rrfs_mpasjedi_2024052700_getkf_solver .....   Passed  1607.45 sec

100% tests passed, 0 tests failed out of 8

Label Time Summary:
mpi            = 3692.56 sec*proc (8 tests)
rdas-bundle    = 3692.56 sec*proc (8 tests)
script         = 3692.56 sec*proc (8 tests)

Total Test time (real) = 2025.01 sec

workdir: /lfs5/BMC/wrfruc/rrfsbot/PRs_RDASApp/238

rrfsbot · 2024-12-01T03:27:17Z

PASSED on hercules

started build_and_test on hercules at UTC time: Sun Dec 1 02:21:14 UTC 2024
finished at UTC time: Sun Dec 1 03:27:17 UTC 2024

Test project /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/238/build/rrfs-test
    Start 2: rrfs_fv3jedi_2024052700_getkf_observer
    Start 5: rrfs_mpasjedi_2024052700_getkf_observer
    Start 1: rrfs_fv3jedi_2024052700_Ens3Dvar
    Start 4: rrfs_mpasjedi_2024052700_Ens3Dvar
    Start 7: rrfs_mpasjedi_2024052700_bumploc
    Start 8: rrfs_bufr2ioda_msonet
1/8 Test #8: rrfs_bufr2ioda_msonet .....................   Passed   49.92 sec
2/8 Test #1: rrfs_fv3jedi_2024052700_Ens3Dvar ..........   Passed  147.48 sec
3/8 Test #2: rrfs_fv3jedi_2024052700_getkf_observer ....   Passed  264.74 sec
    Start 3: rrfs_fv3jedi_2024052700_getkf_solver
4/8 Test #3: rrfs_fv3jedi_2024052700_getkf_solver ......   Passed  415.10 sec
5/8 Test #4: rrfs_mpasjedi_2024052700_Ens3Dvar .........   Passed  719.15 sec
6/8 Test #5: rrfs_mpasjedi_2024052700_getkf_observer ...   Passed  824.07 sec
    Start 6: rrfs_mpasjedi_2024052700_getkf_solver
7/8 Test #7: rrfs_mpasjedi_2024052700_bumploc ..........   Passed  836.35 sec
8/8 Test #6: rrfs_mpasjedi_2024052700_getkf_solver .....   Passed  1245.97 sec

100% tests passed, 0 tests failed out of 8

Label Time Summary:
mpi            = 4502.78 sec*proc (8 tests)
rdas-bundle    = 4502.78 sec*proc (8 tests)
script         = 4502.78 sec*proc (8 tests)

Total Test time (real) = 2070.54 sec

workdir: /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/238

SamuelDegelia-NOAA and others added 14 commits November 20, 2024 03:28

Link in new case and set up expr, fix directories

2ba7713

First update to yamls for new fv3 case

ac4f238

Fix linking script for new yaml files

03a7598

Update yamls for new fix files and layout

85cd906

Remove exit statement from build.sh

efbfbc7

Update yamls for bumploc changes, some renaming

1a51f46

Remove hyb yaml (no static B yet) and update letkf yaml

47d4bc4

Remove hyb ctest and change to 3denvar

9b41a20

Update test reference

e00988c

Change output from hybens to ens for 3denvar

f5577f3

Copy in testoutput for fv3jedi expr dir

857b4b7

Merge branch 'NOAA-EMC:develop' into feature/update_fv3_case

1cad481

Add back old test data

8dfe389

Add tools for running GSI validation for new case

40592c2

SamuelDegelia-NOAA requested review from hu5970, ShunLiu-NOAA, chunhuazhou, delippi, guoqing-noaa and TingLei-NOAA November 27, 2024 21:16

guoqing-noaa reviewed Nov 27, 2024

View reviewed changes

guoqing-noaa previously approved these changes Nov 27, 2024

View reviewed changes

Update to use more obs and GETKF for ctest

385466a

SamuelDegelia-NOAA dismissed guoqing-noaa’s stale review via 385466a November 30, 2024 02:55

Remove old files

35a03b6

SamuelDegelia-NOAA added test_hera test_hercules test_jet labels Nov 30, 2024

rrfsbot added the running_hercules label Nov 30, 2024

rrfsbot added running_jet running_hera Passed_hera and removed test_hercules test_jet test_hera running_hera labels Nov 30, 2024

rrfsbot added Failed_hercules and removed running_hercules labels Nov 30, 2024

rrfsbot added Passed_jet and removed running_jet labels Nov 30, 2024

Update cmake logic for hercules too

5774f91

SamuelDegelia-NOAA added test_hercules and removed Failed_hercules labels Dec 1, 2024

rrfsbot added running_hercules Passed_hercules and removed test_hercules running_hercules labels Dec 1, 2024

ShunLiu-NOAA approved these changes Dec 2, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update FV3-JEDI ctest case to match MPAS-JEDI case #238

Update FV3-JEDI ctest case to match MPAS-JEDI case #238

SamuelDegelia-NOAA commented Nov 27, 2024 •

edited

Loading

guoqing-noaa Nov 27, 2024

guoqing-noaa Nov 27, 2024

guoqing-noaa left a comment

SamuelDegelia-NOAA commented Nov 30, 2024 •

edited

Loading

rrfsbot commented Nov 30, 2024

rrfsbot commented Nov 30, 2024

rrfsbot commented Nov 30, 2024

rrfsbot commented Dec 1, 2024

Update FV3-JEDI ctest case to match MPAS-JEDI case #238

Are you sure you want to change the base?

Update FV3-JEDI ctest case to match MPAS-JEDI case #238

Conversation

SamuelDegelia-NOAA commented Nov 27, 2024 • edited Loading

Description

Issue(s) addressed

Dependencies (if applicable)

Checklist

guoqing-noaa Nov 27, 2024

Choose a reason for hiding this comment

guoqing-noaa Nov 27, 2024

Choose a reason for hiding this comment

guoqing-noaa left a comment

Choose a reason for hiding this comment

SamuelDegelia-NOAA commented Nov 30, 2024 • edited Loading

rrfsbot commented Nov 30, 2024

rrfsbot commented Nov 30, 2024

rrfsbot commented Nov 30, 2024

rrfsbot commented Dec 1, 2024

SamuelDegelia-NOAA commented Nov 27, 2024 •

edited

Loading

SamuelDegelia-NOAA commented Nov 30, 2024 •

edited

Loading